{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HAZNaRRAA_8M"
      },
      "source": [
        "[Hanane D](https://www.linkedin.com/in/hanane-d-algo-trader)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gj9CDzYyB3dk"
      },
      "source": [
        "1- Parsing Amazon 10k Financial Report using:\n",
        "\n",
        "*   **LlamaParse** with **GPT-4o mode** to improve the parsing quality, particularly when financial charts are included in the report\n",
        "\n",
        "*   **SimpleDirectoryParser**: a standard way of parsing.\n",
        "\n",
        "2- I used HuggingFace local **embedding** (using LlamaIndex), to store data in a VectoreStore\n",
        "\n",
        "3- I created a **query engine** using different LLMs and compare their results: **gpt-3.5-turbo**, **GPT-4o**, and **Claude Sonnet 3.5** .\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-ZdBacqLPcMc"
      },
      "source": [
        "# Install Lib"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0vGBK5OdIAzC"
      },
      "outputs": [],
      "source": [
        "!pip install llama-index llama-index-core llama-parse openai llama_index.embeddings.huggingface -q\n",
        "!pip install llama-index-llms-anthropic -q"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UGw8pPODPecc"
      },
      "source": [
        "# Specify API Keys"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "RhRVy7zHIPZN"
      },
      "outputs": [],
      "source": [
        "import nest_asyncio\n",
        "nest_asyncio.apply()\n",
        "\n",
        "from google.colab import userdata\n",
        "OPENAI_API_KEY = userdata.get('OPENAI_API_KEY')\n",
        "LLAMAPARSE_API_KEY = userdata.get('LLAMAPARSE_API_KEY')\n",
        "ANTHROPIC_API_KEY = userdata.get(\"CLAUDE_API_KEY\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g6jbqBtkPn68"
      },
      "source": [
        "# Loading financial report: Amazon 2023 10K"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xX2fkO8-IUwd"
      },
      "outputs": [],
      "source": [
        "!wget \"https://d18rn0p25nwr6d.cloudfront.net/CIK-0001018724/c7c14359-36fa-40c3-b3ca-5bf7f3fa0b96.pdf\" -O amzn_2023_10k.pdf"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2u5EmHwZ0OAb"
      },
      "source": [
        "# Standard Parsing with SimpleDirectoryReader:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0DT9TCMCA23L"
      },
      "source": [
        "## VectoreStore and specifying LLms for the query_engine"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bgcLTa41-nFk",
        "outputId": "bf873e7d-c36e-40e3-8c19-3d6242a274c2"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
            "  warnings.warn(\n"
          ]
        }
      ],
      "source": [
        "# from llama_parse import LlamaParse\n",
        "from llama_index.core import VectorStoreIndex, SimpleDirectoryReader\n",
        "import nest_asyncio;\n",
        "nest_asyncio.apply()\n",
        "\n",
        "pdf_name = \"amzn_2023_10k.pdf\"\n",
        "# use SimpleDirectoryReader to parse our file\n",
        "documents = SimpleDirectoryReader(input_files=[pdf_name]).load_data()\n",
        "from llama_index.core import Settings\n",
        "\n",
        "embed_model = \"local:BAAI/bge-small-en-v1.5\" #https://huggingface.co/collections/BAAI/bge-66797a74476eb1f085c7446d\n",
        "\n",
        "vector_index_std = VectorStoreIndex(documents, embed_model = embed_model)\n",
        "\n",
        "from llama_index.llms.openai import OpenAI\n",
        "\n",
        "llm_gpt35 = OpenAI(model=\"gpt-3.5-turbo\", api_key = OPENAI_API_KEY)\n",
        "query_engine_gpt35 = vector_index_std.as_query_engine(similarity_top_k=3, llm=llm_gpt35)\n",
        "\n",
        "llm_gpt4o = OpenAI(model=\"gpt-4o\", api_key = OPENAI_API_KEY)\n",
        "query_engine_gpt4o = vector_index_std.as_query_engine(similarity_top_k=3, llm=llm_gpt4o)\n",
        "\n",
        "\n",
        "from llama_index.llms.anthropic import Anthropic\n",
        "from llama_index.core import Settings\n",
        "\n",
        "tokenizer = Anthropic().tokenizer\n",
        "Settings.tokenizer = tokenizer\n",
        "\n",
        "llm_claude = Anthropic(model=\"claude-3-5-sonnet-20240620\", api_key=ANTHROPIC_API_KEY)\n",
        "query_engine_claude = vector_index_std.as_query_engine(similarity_top_k=3, llm=llm_claude)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 42,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WhbSMMS2AKRv",
        "outputId": "1b9df5f5-dc89-46cb-cb5b-f79c1493797a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Table of Contents\n",
            "AMAZON.COM, INC.\n",
            "CONSOLIDATED STATEMENTS OF CASH FLOWS\n",
            "(in millions)\n",
            "  Year Ended December 31,\n",
            " 2021 2022 2023\n",
            "CASH, CASH EQUIV ALENTS, AND RESTRICTED CASH, BEGINNING OF PERIOD $ 42,377 $ 36,477 $ 54,253 \n",
            "OPERA TING ACTIVITIES:\n",
            "Net income (loss) 33,364 (2,722) 30,425 \n",
            "Adjustments to reconcile net income (loss) to net cash from operating activities:\n",
            "Depreciation and amortization of property and equipment and capitalized content costs, operating lease\n",
            "assets, and other 34,433 41,921 48,663 \n",
            "Stock-based compensation 12,757 19,621 24,023 \n",
            "Non-operating expense (income), net (14,306) 16,966 (748)\n",
            "Deferred income taxes (310) (8,148) (5,876)\n",
            "Changes in operating assets and liabilities:\n",
            "Inventories (9,487) (2,592) 1,449 \n",
            "Accounts receivable, net and other (9,145) (8,622) (8,348)\n",
            "Other assets (9,018) (13,275) (12,265)\n",
            "Accounts payable 3,602 2,945 5,473 \n",
            "Accrued expenses and other 2,123 (1,558) (2,428)\n",
            "Unearned revenue 2,314 2,216 4,578 \n",
            "Net cash provided by (used in) operating activities 46,327 46,752 84,946 \n",
            "INVESTING ACTIVITIES:\n",
            "Purchases of property and equipment (61,053) (63,645) (52,729)\n",
            "Proceeds from property and equipment sales and incentives 5,657 5,324 4,596 \n",
            "Acquisitions, net of cash acquired, non-marketable investments, and other (1,985) (8,316) (5,839)\n",
            "Sales and maturities of marketable securities 59,384 31,601 5,627 \n",
            "Purchases of marketable securities (60,157) (2,565) (1,488)\n",
            "Net cash provided by (used in) investing activities (58,154) (37,601) (49,833)\n",
            "FINANCING ACTIVITIES:\n",
            "Common stock repurchased — (6,000) — \n",
            "Proceeds from short-term debt, and other 7,956 41,553 18,129 \n",
            "Repayments of short-term debt, and other (7,753) (37,554) (25,677)\n",
            "Proceeds from long-term debt 19,003 21,166 — \n",
            "Repayments of long-term debt (1,590) (1,258) (3,676)\n",
            "Principal repayments of finance leases (11,163) (7,941) (4,384)\n",
            "Principal repayments of financing obligations (162) (248) (271)\n",
            "Net cash provided by (used in) financing activities 6,291 9,718 (15,879)\n",
            "Foreign currency ef fect on cash, cash equivalents, and restricted cash (364) (1,093) 403 \n",
            "Net increase (decrease) in cash, cash equivalents, and restricted cash (5,900) 17,776 19,637 \n",
            "CASH, CASH EQUIV ALENTS, AND RESTRICTED CASH, END OF PERIOD $ 36,477 $ 54,253 $ 73,890 \n",
            "See accompanying notes to consolidated financial statements.\n",
            "37\n"
          ]
        }
      ],
      "source": [
        "print(documents[36].text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MqxqSvuIARon"
      },
      "source": [
        "## Chatting with the LLMs: GPT-3.5-Turbo, GPT-4o, Claude 3.5 Sonnet"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 43,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6fHkpLIM-CNB",
        "outputId": "c037365b-bb7a-465c-a573-23f8d2c2c554"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "GPT-3.5 Turbo\n",
            "The net income for 2023 is $36,813 million.\n",
            "\n",
            "GPT-4o\n",
            "The net income for 2023 is not explicitly provided in the given context. However, you can calculate it using the provided information. The income before income taxes for 2023 is $37,557 million, and the provision for income taxes is $7,120 million. Therefore, the net income for 2023 would be $37,557 million minus $7,120 million, which equals $30,437 million.\n",
            "\n",
            "Claude 3.5 Sonnet\n",
            "Based on the information provided, Amazon's net income for 2023 can be calculated as $30,437 million. This is derived from the \"Income (loss) before income taxes\" of $37,557 million for 2023, minus the \"Provision (benefit) for income taxes, net\" of $7,120 million for the same year.\n"
          ]
        }
      ],
      "source": [
        "query1 = \"What was the net income in 2023?\"\n",
        "response = query_engine_gpt35.query(query1)\n",
        "print(\"GPT-3.5 Turbo\")\n",
        "print(str(response))\n",
        "\n",
        "resp = query_engine_gpt4o.query(query1)\n",
        "print(\"\\nGPT-4o\")\n",
        "print(str(resp))\n",
        "\n",
        "resp = query_engine_claude.query(query1)\n",
        "print(\"\\nClaude 3.5 Sonnet\")\n",
        "print(str(resp))\n",
        "\n",
        "# print(resp.source_nodes[0].get_content()) # to get the source_node used to answer"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 44,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zwRqMiJTBJup",
        "outputId": "8257c1bf-34ab-46e6-870d-efe69cfe5da2"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "GPT-3.5 Turbo\n",
            "The net income for 2022 is a provision (benefit) for income taxes of $(3.2) billion.\n",
            "\n",
            "GPT-4o\n",
            "The net income for 2022 is not explicitly provided in the given context. However, you can calculate it using the provided information. The income (loss) before income taxes for 2022 is $(5,936) million, and the provision (benefit) for income taxes is $(3,217) million. Therefore, the net income for 2022 would be:\n",
            "\n",
            "Net Income = Income (Loss) Before Income Taxes - Provision (Benefit) for Income Taxes\n",
            "Net Income = $(5,936) million - $(3,217) million\n",
            "Net Income = $(5,936) million + $3,217 million\n",
            "Net Income = $(2,719) million\n",
            "\n",
            "So, the net income for 2022 is $(2,719) million.\n",
            "\n",
            "Claude 3.5 Sonnet\n",
            "Based on the information provided, Amazon reported a net loss in 2022. The company had a loss before income taxes of $5,936 million in 2022. After accounting for an income tax benefit of $3,217 million that year, the net loss for 2022 would be approximately $2,719 million.\n"
          ]
        }
      ],
      "source": [
        "query1 = \"What was the net income in 2022?\"\n",
        "response = query_engine_gpt35.query(query1)\n",
        "print(\"GPT-3.5 Turbo\")\n",
        "print(str(response))\n",
        "\n",
        "resp = query_engine_gpt4o.query(query1)\n",
        "print(\"\\nGPT-4o\")\n",
        "print(str(resp))\n",
        "\n",
        "resp = query_engine_claude.query(query1)\n",
        "print(\"\\nClaude 3.5 Sonnet\")\n",
        "print(str(resp))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 46,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sFa3nnoR1Qye",
        "outputId": "40b124e5-65b8-4bbf-ffe1-5f230eda1940"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "query2: What was the net income in 2023 compared to 2022?\n",
            "\n",
            "GPT-3.5 Turbo\n",
            "The net income in 2023 was $37.557 billion, while in 2022, it was $(5.936) billion.\n",
            "\n",
            "GPT-4o\n",
            "The net income in 2023 was higher compared to 2022. In 2023, the income before income taxes was $37,557 million, and the provision for income taxes was $7,120 million, resulting in a net income of $30,437 million. In contrast, in 2022, the loss before income taxes was $5,936 million, and the benefit for income taxes was $3,217 million, resulting in a net loss of $2,719 million.\n",
            "\n",
            "Claude 3.5 Sonnet\n",
            "Based on the financial information provided, Amazon's net income situation changed significantly from 2022 to 2023:\n",
            "\n",
            "In 2022, Amazon had a loss before income taxes of $5,936 million and received an income tax benefit of $3,217 million.\n",
            "\n",
            "In 2023, Amazon had income before income taxes of $37,557 million and incurred an income tax provision of $7,120 million.\n",
            "\n",
            "While the exact net income figure isn't directly stated, we can infer that Amazon went from a net loss position in 2022 to a substantial net profit in 2023. The dramatic increase in pre-tax income of over $43 billion year-over-year, combined with returning to paying taxes rather than receiving a tax benefit, indicates a major improvement in Amazon's profitability in 2023 compared to 2022.\n"
          ]
        }
      ],
      "source": [
        "query2 = \"What was the net income in 2023 compared to 2022?\"\n",
        "print(\"query2:\",query2)\n",
        "response = query_engine_gpt35.query(query2)\n",
        "print(\"\\nGPT-3.5 Turbo\")\n",
        "print(str(response))\n",
        "\n",
        "resp = query_engine_gpt4o.query(query2)\n",
        "print(\"\\nGPT-4o\")\n",
        "print(str(resp))\n",
        "\n",
        "resp = query_engine_claude.query(query2)\n",
        "print(\"\\nClaude 3.5 Sonnet\")\n",
        "print(str(resp))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 47,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LWve5kAw_bfV",
        "outputId": "a260dadc-5b3d-426b-8546-70b13fdecc7e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Table of Contents\n",
            "The components of the provision (benefit) for income taxes, net are as follows (in millions):\n",
            " Year Ended December 31,\n",
            "2021 2022 2023\n",
            "U.S. Federal:\n",
            "Current $ 2,129 $ 2,175 $ 8,652 \n",
            "Deferred 155 (6,686) (5,505)\n",
            "Total 2,284 (4,511) 3,147 \n",
            "U.S. State:\n",
            "Current 763 1,074 2,158 \n",
            "Deferred (178) (1,302) (498)\n",
            "Total 585 (228) 1,660 \n",
            "International:\n",
            "Current 2,209 1,682 2,186 \n",
            "Deferred (287) (160) 127 \n",
            "Total 1,922 1,522 2,313 \n",
            "Provision (benefit) for income taxes, net $ 4,791 $ (3,217)$ 7,120 \n",
            "U.S. and international components of income (loss) before income taxes are as follows (in millions):\n",
            " Year Ended December 31,\n",
            " 2021 2022 2023\n",
            "U.S. $ 35,879 $ (8,225)$ 32,328 \n",
            "International 2,272 2,289 5,229 \n",
            "Income (loss) before income taxes $ 38,151 $ (5,936)$ 37,557 \n",
            "The items accounting for differences between income taxes computed at the federal statutory rate and the provision (benefit) recorded for income taxes\n",
            "are as follows (in millions):\n",
            " Year Ended December 31,\n",
            " 2021 2022 2023\n",
            "Income taxes computed at the federal statutory rate $ 8,012 $ (1,246)$ 7,887 \n",
            "Effect of:\n",
            "Tax impact of foreign earnings and losses (1,349) (370) 594 \n",
            "State taxes, net of federal benefits 465 (173) 1,307 \n",
            "Tax credits (1,136) (1,006) (2,362)\n",
            "Stock-based compensation (1) (1,094) 612 1,047 \n",
            "Foreign income deduction (2) (301) (1,258) (1,429)\n",
            "Other, net 194 224 76 \n",
            "Total $ 4,791 $ (3,217)$ 7,120 \n",
            "___________________\n",
            "(1)Includes non-deductible stock-based compensation and excess tax benefits or shortfalls from stock-based compensation. Our tax provision includes $1.9\n",
            "billion of excess tax benefits from stock-based compensation for 2021, and $33 million and $519 million of tax shortfalls from stock-based compensation\n",
            "for 2022 and 2023.\n",
            "(2)U.S. companies are eligible for a deduction that lowers the effective tax rate on certain foreign income. This regime is referred to as the Foreign-Derived\n",
            "Intangible Income deduction and is dependent on the amount of our U.S. taxable income.\n",
            "We generated an income tax benefit in 2022 as compared to a provision for income taxes in 2021 primarily due to a decrease in pretax income and an\n",
            "increase in the foreign income deduction. This was partially offset by a reduction in excess tax benefits from stock-based compensation and a decrease in the\n",
            "tax impact of foreign earnings and losses driven by a decline in the favorable effects of corporate restructuring transactions. The foreign income deduction\n",
            "benefit recognized in 2022 reflects a change in our application of tax regulations related to the computation of qualifying foreign income and includes a tax\n",
            "benefit of approximately $655 million related to years prior to 2022.\n",
            "65\n"
          ]
        }
      ],
      "source": [
        "print(resp.source_nodes[0].get_content())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "INxsH6xbzv48"
      },
      "source": [
        "# LlamaParse: Amazon 10K Financial report"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tgTPlUu4Ln8U",
        "outputId": "c5748a2e-39c4-4ff5-c038-8c3a52579100"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Started parsing the file under job_id cac11eca-3194-4b60-a73f-76fb55d0bbaa\n"
          ]
        }
      ],
      "source": [
        "from llama_parse import LlamaParse\n",
        "from llama_index.core import VectorStoreIndex\n",
        "\n",
        "pdf_name = \"amzn_2023_10k.pdf\"\n",
        "# set up parser\n",
        "# parser = LlamaParse(api_key=LLAMAPARSE_API_KEY, result_type=\"markdown\", gpt4o_mode = True)\n",
        "parser = LlamaParse(api_key=LLAMAPARSE_API_KEY, result_type=\"markdown\", gpt4o_mode = True)\n",
        "documents = parser.load_data(pdf_name)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "J6RUnjbrLzyu",
        "outputId": "25c1732d-41e8-4209-81fd-5699d07c0ea7"
      },
      "outputs": [],
      "source": [
        "parser"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5qc4mNShOmJd",
        "outputId": "f6b04896-7c37-45f1-ad2f-787104714f0b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "# AMAZON.COM, INC.\n",
            "## CONSOLIDATED STATEMENTS OF CASH FLOWS\n",
            "### (in millions)\n",
            "#### Year Ended December 31,\n",
            "\n",
            "|                                | 2021     | 2022     | 2023     |\n",
            "|--------------------------------|----------|----------|----------|\n",
            "| **CASH, CASH EQUIVALENTS, AND RESTRICTED CASH, BEGINNING OF PERIOD** | $ 42,377 | $ 36,477 | $ 54,253 |\n",
            "| **OPERATING ACTIVITIES:**      |          |          |          |\n",
            "| Net income (loss)              | 33,364   | (2,722)  | 30,425   |\n",
            "| Adjustments to reconcile net income (loss) to net cash from operating activities: |          |          |          |\n",
            "| Depreciation and amortization of property and equipment and capitalized content costs, operating lease assets, and other | 34,433   | 41,921   | 48,663   |\n",
            "| Stock-based compensation       | 12,757   | 19,621   | 24,023   |\n",
            "| Non-operating expense (income), net | (14,306)  | 16,966   | (748)    |\n",
            "| Deferred income taxes          | (310)    | (8,145)  | (5,876)  |\n",
            "| Changes in operating assets and liabilities: |          |          |          |\n",
            "| Inventories                    | (9,487)  | (2,592)  | 1,449    |\n",
            "| Accounts receivable, net and other | (9,145)  | (8,822)  | 2,345    |\n",
            "| Other assets                   | (9,018)  | (13,275) | (12,365) |\n",
            "| Accounts payable               | 3,602    | 2,945    | 5,473    |\n",
            "| Accrued expenses and other     | 3,123    | (1,558)  | (2,473)  |\n",
            "| Unearned revenue               | 2,314    | 2,210    | 4,798    |\n",
            "| Net cash provided by (used in) operating activities | 46,327   | 46,752   | 84,946   |\n",
            "| **INVESTING ACTIVITIES:**      |          |          |          |\n",
            "| Purchases of property and equipment | (61,053)  | (63,454) | (52,729) |\n",
            "| Proceeds from property and equipment sales and incentives | 5,657    | 5,324    | 4,596    |\n",
            "| Acquisitions, net of cash acquired, non-marketable investments, and other | (1,985)   | (8,316)  | (5,839)  |\n",
            "| Sales and maturities of marketable securities | 59,384   | 31,101   | 43,527   |\n",
            "| Purchases of marketable securities | (60,157)  | (2,565)  | (1,488)  |\n",
            "| Net cash provided by (used in) investing activities | (58,154)  | (37,901) | (49,933) |\n",
            "| **FINANCING ACTIVITIES:**      |          |          |          |\n",
            "| Common stock repurchased       | —        | (6,000)  | —        |\n",
            "| Proceeds from short-term debt, and other | 7,956    | 41,553   | 18,129   |\n",
            "| Repayments of short-term debt, and other | (7,753)   | (37,554) | (25,877) |\n",
            "| Proceeds from long-term debt   | 19,003   | 21,166   | 18,129   |\n",
            "| Repayments of long-term debt   | (1,590)  | (1,258)  | (3,766)  |\n",
            "| Principal repayments of finance leases | (11,163)  | (7,941)  | (4,384)  |\n",
            "| Principal repayments of financing obligations | (162)    | (248)    | (271)    |\n",
            "| Net cash provided by (used in) financing activities | 6,291    | 9,718    | (15,979) |\n",
            "| Foreign currency effect on cash, cash equivalents, and restricted cash | (364)    | (1,093)  | 403      |\n",
            "| Net increase (decrease) in cash, cash equivalents, and restricted cash | (5,900)  | 17,576   | 19,637   |\n",
            "| **CASH, CASH EQUIVALENTS, AND RESTRICTED CASH, END OF PERIOD** | $ 36,477 | $ 54,253 | $ 73,890 |\n",
            "\n",
            "See accompanying notes to consolidated financial statements.\n"
          ]
        }
      ],
      "source": [
        "print(documents[43].text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v1138pKFSmck"
      },
      "source": [
        "## Store documents: with embeddings for later retrieval"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "C82-nRDnOay0",
        "outputId": "6d7bc1b2-3f29-46d3-8d05-244eb7530e46"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
            "  warnings.warn(\n"
          ]
        }
      ],
      "source": [
        "from llama_index.core import VectorStoreIndex\n",
        "\n",
        "embed_model = \"local:BAAI/bge-small-en-v1.5\" #https://huggingface.co/collections/BAAI/bge-66797a74476eb1f085c7446d\n",
        "vector_index = VectorStoreIndex(documents, embed_model = embed_model)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "bxCWYnMn3tdS"
      },
      "outputs": [],
      "source": [
        "# vector_index.storage_context.persist(persist_dir=path)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mQmNEzxFAndN"
      },
      "source": [
        "## Specify LLMs for chat"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "id": "RnUR1tZH2qYB"
      },
      "outputs": [],
      "source": [
        "from llama_index.llms.openai import OpenAI\n",
        "\n",
        "llm_gpt35 = OpenAI(model=\"gpt-3.5-turbo\", api_key = OPENAI_API_KEY)\n",
        "query_engine_gpt35 = vector_index.as_query_engine(similarity_top_k=3, llm=llm_gpt35)\n",
        "\n",
        "llm_gpt4o = OpenAI(model=\"gpt-4o\", api_key = OPENAI_API_KEY)\n",
        "query_engine_gpt4o = vector_index.as_query_engine(similarity_top_k=3, llm=llm_gpt4o)\n",
        "\n",
        "from llama_index.llms.anthropic import Anthropic\n",
        "from llama_index.core import Settings\n",
        "\n",
        "tokenizer = Anthropic().tokenizer\n",
        "Settings.tokenizer = tokenizer\n",
        "\n",
        "llm_claude = Anthropic(model=\"claude-3-5-sonnet-20240620\", api_key=ANTHROPIC_API_KEY)\n",
        "query_engine_claude = vector_index.as_query_engine(similarity_top_k=3, llm=llm_claude)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3ez8VxWQAqyO"
      },
      "source": [
        "## Chatting with the LLMs: GPT-3.5-Turbo, GPT-4o, Claude 3.5 Sonnet"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ly6_5ovJ4KSI",
        "outputId": "b3d9c39c-e5c1-4d98-a385-f54db99f37c5"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "GPT-3.5 Turbo\n",
            "The net income for 2023 is $16.9 billion.\n",
            "\n",
            "GPT-4o\n",
            "The net income for 2023 can be calculated by subtracting the provision for income taxes from the income before income taxes. For 2023, the income before income taxes is $37,557 million, and the provision for income taxes is $7,120 million. Therefore, the net income for 2023 is $37,557 million - $7,120 million = $30,437 million.\n",
            "\n",
            "Claude 3.5 Sonnet\n",
            "The net income for 2023 can be calculated by subtracting the provision for income taxes from the income before income taxes.\n",
            "\n",
            "Income before income taxes in 2023 was $37,557 million.\n",
            "Provision for income taxes in 2023 was $7,120 million.\n",
            "\n",
            "Therefore, the net income for 2023 was:\n",
            "$37,557 million - $7,120 million = $30,437 million\n",
            "\n",
            "So, the net income for 2023 was $30,437 million.\n"
          ]
        }
      ],
      "source": [
        "query1 = \"What is the net income on 2023?\"\n",
        "response = query_engine_gpt35.query(query1)\n",
        "print(\"GPT-3.5 Turbo\")\n",
        "print(str(response))\n",
        "\n",
        "resp = query_engine_gpt4o.query(query1)\n",
        "print(\"\\nGPT-4o\")\n",
        "print(str(resp))\n",
        "\n",
        "resp = query_engine_claude.query(query1)\n",
        "print(\"\\nClaude 3.5 Sonnet\")\n",
        "print(str(resp))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GDXx83t6S75z",
        "outputId": "38af063f-0f18-4a16-c650-1ab29f2ab33f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "query2: What was the net income in 2023 compared to 2022?\n",
            "\n",
            "GPT-3.5 Turbo\n",
            "The net income in 2023 was higher compared to 2022.\n",
            "\n",
            "GPT-4o\n",
            "The net income in 2023 was $30.4 billion, compared to a net loss of $2.7 billion in 2022.\n",
            "\n",
            "Claude 3.5 Sonnet\n",
            "The net income in 2023 was significantly higher compared to 2022. In 2023, the company reported income before income taxes of $37.557 billion, and after accounting for income taxes of $7.120 billion, the net income would be approximately $30.437 billion. \n",
            "\n",
            "In contrast, 2022 saw a loss before income taxes of $5.936 billion. However, there was a tax benefit of $3.217 billion, which would result in a net loss of approximately $2.719 billion for 2022.\n",
            "\n",
            "This represents a substantial turnaround, with the company moving from a net loss position in 2022 to a significant profit in 2023. The improvement was primarily driven by a large increase in income before taxes, reflecting better overall business performance in 2023.\n"
          ]
        }
      ],
      "source": [
        "query2 = \"What was the net income in 2023 compared to 2022?\"\n",
        "print(\"query2:\",query2)\n",
        "response = query_engine_gpt35.query(query2)\n",
        "print(\"\\nGPT-3.5 Turbo\")\n",
        "print(str(response))\n",
        "\n",
        "resp = query_engine_gpt4o.query(query2)\n",
        "print(\"\\nGPT-4o\")\n",
        "print(str(resp))\n",
        "\n",
        "resp = query_engine_claude.query(query2)\n",
        "print(\"\\nClaude 3.5 Sonnet\")\n",
        "print(str(resp))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UiiBBhOKTGwr",
        "outputId": "18a1ea4c-cc23-4f6f-c81c-7ebe0e35cc42"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "query2: What was the reason of the net income loss in 2022?\n",
            "\n",
            "GPT-3.5 Turbo\n",
            "The net income loss in 2022 was primarily due to a decrease in pretax income, an increase in the foreign income deduction, a reduction in excess tax benefits from stock-based compensation, and a decrease in the tax impact of foreign earnings and losses driven by a decline in the favorable effects of corporate restructuring transactions.\n",
            "\n",
            "GPT-4o\n",
            "The net income loss in 2022 was primarily due to a decrease in pretax income and an increase in the foreign income deduction. This was partially offset by a reduction in excess tax benefits from stock-based compensation and a decrease in the tax impact of foreign earnings and losses driven by a decline in the favorable effects of corporate restructuring transactions.\n",
            "\n",
            "Claude 3.5 Sonnet\n",
            "The net income loss in 2022 was primarily due to a significant decrease in pretax income. The company experienced a loss before income taxes of $5,936 million in 2022, compared to an income of $38,151 million in 2021. This substantial decline in pretax income was a major factor contributing to the net loss.\n",
            "\n",
            "Additionally, there were other factors that influenced the tax situation in 2022:\n",
            "\n",
            "1. An increase in the foreign income deduction, which lowered the effective tax rate on certain foreign income.\n",
            "\n",
            "2. A reduction in excess tax benefits from stock-based compensation.\n",
            "\n",
            "3. A decrease in the tax impact of foreign earnings and losses, driven by a decline in the favorable effects of corporate restructuring transactions.\n",
            "\n",
            "4. The company also recognized a change in the application of tax regulations related to the computation of qualifying foreign income, which included a tax benefit of approximately $655 million for years prior to 2022.\n",
            "\n",
            "These factors combined to result in an income tax benefit of $3,217 million in 2022, as opposed to a tax provision in the previous year. However, this tax benefit was not sufficient to offset the significant decrease in pretax income, leading to the overall net loss for 2022.\n"
          ]
        }
      ],
      "source": [
        "query2 = \"What was the reason of the net income loss in 2022?\"\n",
        "print(\"query2:\",query2)\n",
        "response = query_engine_gpt35.query(query2)\n",
        "print(\"\\nGPT-3.5 Turbo\")\n",
        "print(str(response))\n",
        "\n",
        "resp = query_engine_gpt4o.query(query2)\n",
        "print(\"\\nGPT-4o\")\n",
        "print(str(resp))\n",
        "\n",
        "resp = query_engine_claude.query(query2)\n",
        "print(\"\\nClaude 3.5 Sonnet\")\n",
        "print(str(resp))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bzfHzw2_TU8E",
        "outputId": "c23d0bf3-db6e-4d64-e0de-5b966f1b518e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "# The components of the provision (benefit) for income taxes, net are as follows (in millions):\n",
            "\n",
            "|                            | Year Ended December 31, |\n",
            "|----------------------------|-------------------------|\n",
            "|                            | 2021    | 2022    | 2023    |\n",
            "| **U.S. Federal:**          |         |         |         |\n",
            "| Current                    | $ 2,129 | $ 2,175 | $ 8,652 |\n",
            "| Deferred                   | 155     | (6,686) | (5,505) |\n",
            "| **Total**                  | 2,284   | (4,511) | 3,147   |\n",
            "| **U.S. State:**            |         |         |         |\n",
            "| Current                    | 763     | 1,074   | 2,158   |\n",
            "| Deferred                   | (178)   | (1,302) | (498)   |\n",
            "| **Total**                  | 585     | (228)   | 1,660   |\n",
            "| **International:**         |         |         |         |\n",
            "| Current                    | 2,209   | 1,682   | 2,186   |\n",
            "| Deferred                   | (287)   | (160)   | 127     |\n",
            "| **Total**                  | 1,922   | 1,522   | 2,313   |\n",
            "| **Provision (benefit) for income taxes, net** | $ 4,791 | $ (3,217) | $ 7,120 |\n",
            "\n",
            "# U.S. and international components of income (loss) before income taxes are as follows (in millions):\n",
            "\n",
            "|                            | Year Ended December 31, |\n",
            "|----------------------------|-------------------------|\n",
            "|                            | 2021    | 2022    | 2023    |\n",
            "| **U.S.**                   | $ 35,579 | $ (8,225) | $ 32,328 |\n",
            "| **International**          | 2,272   | 2,289   | 5,229   |\n",
            "| **Income (loss) before income taxes** | $ 38,151 | $ (5,936) | $ 37,557 |\n",
            "\n",
            "# The items accounting for differences between income taxes computed at the federal statutory rate and the provision (benefit) recorded for income taxes are as follows (in millions):\n",
            "\n",
            "|                            | Year Ended December 31, |\n",
            "|----------------------------|-------------------------|\n",
            "|                            | 2021    | 2022    | 2023    |\n",
            "| **Income taxes computed at the federal statutory rate** | $ 8,012 | $ (1,246) | $ 7,857 |\n",
            "| **Effect of:**             |         |         |         |\n",
            "| Tax impact of foreign earnings and losses | (1,349) | (370)   | 594     |\n",
            "| State taxes, net of federal benefits | 465     | 173     | 1,307   |\n",
            "| Tax credits                | (1,136) | (1,006) | (2,362) |\n",
            "| Stock-based compensation (1) | (1,094) | 612     | 1,047   |\n",
            "| Foreign income deduction (2) | (301)   | (1,258) | (1,429) |\n",
            "| Other, net                 | 194     | 224     | 76      |\n",
            "| **Total**                  | $ 4,791 | $ (3,217) | $ 7,120 |\n",
            "\n",
            "1. Includes non-deductible stock-based compensation and excess tax benefits or shortfalls from stock-based compensation. Our tax provision includes $1.9 billion of excess tax benefits from stock-based compensation for 2021, and $33 million and $519 million of tax shortfalls from stock-based compensation for 2022 and 2023.\n",
            "2. U.S. companies are eligible for a deduction that lowers the effective tax rate on certain foreign income. This regime is referred to as the Foreign-Derived Intangible Income deduction and is dependent on the amount of our U.S. taxable income.\n",
            "\n",
            "We generated an income tax benefit in 2022 as compared to a provision for income taxes in 2021 primarily due to a decrease in pretax income and an increase in the foreign income deduction. This was partially offset by a reduction in excess tax benefits from stock-based compensation and a decrease in the tax impact of foreign earnings and losses driven by a decline in the favorable effects of corporate restructuring transactions. The foreign income deduction benefit recognized in 2022 reflects a change in our application of tax regulations related to the computation of qualifying foreign income and includes a tax benefit of approximately $655 million related to years prior to 2022.\n"
          ]
        }
      ],
      "source": [
        "print(resp.source_nodes[0].get_content())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cBLk8WYR7T0L",
        "outputId": "9f124a71-1e0a-4a4d-e514-7a5c0ce71732"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "query3: What are the most important takeaways from the report?\n",
            "\n",
            "GPT-3.5 Turbo\n",
            "The report includes the consolidated financial statements, notes to the financial statements, and the independent auditor's report. Additionally, it lists the documents filed as part of the report, including the index to consolidated financial statements, financial statement schedules, and exhibits. The forward-looking statements section highlights the uncertainties and factors that could impact future results.\n",
            "\n",
            "GPT-4o\n",
            "The most important takeaways from the report include the detailed financial statements and supplementary data, such as the consolidated statements of cash flows, operations, comprehensive income (loss), balance sheets, and stockholders' equity. Additionally, the report includes the independent auditor's report from Ernst & Young LLP. The management's discussion and analysis section highlights the company's forward-looking statements, emphasizing the inherent uncertainties and various factors that could affect future results, such as fluctuations in foreign exchange rates, changes in global economic conditions, customer demand, and supply chain disruptions. The report also lists several exhibits, including the company's amended and restated certificate of incorporation and bylaws, as well as various indentures and officers' certificates related to the company's notes.\n",
            "\n",
            "Claude 3.5 Sonnet\n",
            "The key takeaways from this report include:\n",
            "\n",
            "1. The company's financial statements are audited by Ernst & Young LLP, an independent registered public accounting firm.\n",
            "\n",
            "2. The report contains consolidated financial statements covering cash flows, operations, comprehensive income/loss, balance sheets, and stockholders' equity for multiple years.\n",
            "\n",
            "3. There are several exhibits referenced, including amended and restated certificates of incorporation and bylaws, as well as various indentures and notes.\n",
            "\n",
            "4. The company acknowledges that its future results and financial position may be affected by various factors, including foreign exchange rates, economic conditions, customer demand, inflation, and supply chain disruptions.\n",
            "\n",
            "5. The company expects variability in its accounts payable days due to factors like product sales mix, use of third-party sellers, inventory management strategies, and investments in new markets and product lines.\n",
            "\n",
            "6. The report contains forward-looking statements that reflect management's expectations but are inherently uncertain and subject to various risks and factors that could cause actual results to differ.\n",
            "\n",
            "7. The company's financial performance and operations are influenced by a complex mix of factors, including its product and service offerings, sales channels, fulfillment strategies, and global expansion efforts.\n"
          ]
        }
      ],
      "source": [
        "query3 = \"What are the most important takeaways from the report?\"\n",
        "print(\"query3:\",query3)\n",
        "response = query_engine_gpt35.query(query3)\n",
        "print(\"\\nGPT-3.5 Turbo\")\n",
        "print(str(response))\n",
        "\n",
        "resp = query_engine_gpt4o.query(query3)\n",
        "print(\"\\nGPT-4o\")\n",
        "print(str(resp))\n",
        "\n",
        "resp = query_engine_claude.query(query3)\n",
        "print(\"\\nClaude 3.5 Sonnet\")\n",
        "print(str(resp))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Zydwxxcs8Non"
      },
      "outputs": [],
      "source": [
        "# For this question, the cost went from $0.07 to $0.16 ==> $0.09"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QReqlaQV7z9y",
        "outputId": "09a93ff2-ce80-4050-aac3-9c624a6583c9"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "# Item 8. Financial Statements and Supplementary Data\n",
            "\n",
            "## INDEX TO CONSOLIDATED FINANCIAL STATEMENTS\n",
            "\n",
            "|                                                                                           | Page |\n",
            "|-------------------------------------------------------------------------------------------|------|\n",
            "| Report of Ernst & Young LLP, Independent Registered Public Accounting Firm (PCAOB ID: 42) | 35   |\n",
            "| Consolidated Statements of Cash Flows                                                     | 37   |\n",
            "| Consolidated Statements of Operations                                                     | 38   |\n",
            "| Consolidated Statements of Comprehensive Income (Loss)                                    | 39   |\n",
            "| Consolidated Balance Sheets                                                               | 40   |\n",
            "| Consolidated Statements of Stockholders’ Equity                                           | 41   |\n",
            "| Notes to Consolidated Financial Statements                                                | 42   |\n"
          ]
        }
      ],
      "source": [
        "print(resp.source_nodes[0].get_content())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bBeqL4y172sz"
      },
      "outputs": [],
      "source": [
        "print(resp.source_nodes[1].get_content())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZZ5v5yYQ77nS"
      },
      "outputs": [],
      "source": [
        "print(resp.source_nodes[2].get_content())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MFnsIP_PcMOz"
      },
      "source": [
        "# Key Takeaways"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Km8ZGwuPcOsY"
      },
      "source": [
        "1- **In terms of parsing**, since the 10k report includes only tables, using LlamaParse with markdown is sufficient (and also visually clear). There's no need for GPT-4o mode. However, as you don't know if there are charts in the PDFs, using GPT-4o mode could be beneficial. However, I found that Claude 3.5 Sonnet (in a previous post) was better than GPT-4o on parsing charts as images.\n",
        "\n",
        "It could be great to have Claude 3.5 Sonnet mode on LlamaParse. It can also be more expensive.\n",
        "\n",
        "The simple parsing (SimpleDirectoryReader), was good too for these kind of simple questions.\n",
        "\n",
        "2- **In the chat comparison**, Claude 3.5 Sonnet was slightly better than GPT-4o. It provides accurate answers and offers valuable details on how some calculations are performed, as well as justifications for certain losses. However, GPT-3.5-Turbo lags significantly behind."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
